Feature-based Transfer Learning Research Articles

A few-shot based phase-batch multi-layer domain adaptation pattern recognition method

Deep transfer learning has been widely applied in the field of intelligent fault diagnosis. However, existing deep transfer learning-based diagnostic methods struggle to train reliable diagnostic models when there is a lack of data and significant distribution differences between the two domains. To address this issue, a few-shot based phase-batch multi-layer domain adaptation pattern recognition method is proposed. This method simultaneously measures the feature distribution differences of both the fully connected layers and the classification layers, thus better correcting the data domain bias. Additionally, a phase-batch training strategy and pseudo-label learning are employed to improve the convergence speed and stability of the training process. The proposed method is validated on two public datasets, Jiang Nan and Paderborn University, as well as a dataset obtained through independent experiments. It is compared with traditional feature-based transfer learning methods, the results show that the proposed method achieves higher diagnostic accuracy, faster convergence, and greater stability. Furthermore, its superior diagnostic performance in the few-shot scenario is demonstrated through experiments on a self-collected dataset.

Identification of multiclass tympanic membranes by using deep feature transfer learning and hyperparameter optimization

Middle ear health is a process that generally depends on eardrum health. Middle ear disorders are more common during childhood. Permanent damage may occur in bacterial or viral infections in this region if an early diagnosis is not made. Infectious ear disease, especially known as Otitis Media, is one of the diseases. In this study, morphological features of images are obtained by using various feature extraction methods. Deep feature-based transfer learning and hyperparameter optimization methods were used to detect the presence and type of otitis media. While the EfficientNet convolutional neural network (CNN) model was used to extract deep features, KNN, SVM, and Ensemble classifiers were used as classifiers. Bayesian, Grid Search, and Random Search were used for hyperparameter optimization. As a result of the experiments carried out, it was observed that the classification performance was 99.1%.

Rapid quantitative analysis of raw rocks by LIBS coupled with feature-based transfer learning

Novel LIBS system with machine vision streamlines on-site elemental analysis in raw rocks, applying transfer learning for elemental prediction and eliminating the need for lab testing. Valuable for rapid field assessments and industrial applications.

Multitree Genetic Programming With Feature-Based Transfer Learning for Symbolic Regression on Incomplete Data.

Data incompleteness is a serious challenge in real-world machine-learning tasks. Nevertheless, it has not received enough attention in symbolic regression (SR). Data missingness exacerbates data shortage, especially in domains with limited available data, which in turn limits the learning ability of SR algorithms. Transfer learning (TL), which aims to transfer knowledge across tasks, is a potential solution to solve this issue by making amends for the lack of knowledge. However, this approach has not been adequately investigated in SR. To fill this gap, a multitree genetic programming-based TL method is proposed in this work to transfer knowledge from complete source domains (SDs) to incomplete related target domains (TDs). The proposed method transforms the features from a complete SD to an incomplete TD. However, having many features complicates the transformation process. To mitigate this problem, we integrate a feature selection mechanism to eliminate unnecessary transformations. The method is examined on real-world and synthetic SR tasks with missing values to consider different learning scenarios. The obtained results not only show the effectiveness of the proposed method but also show its training efficiency compared with the existing TL methods. Compared to state-of-the-art methods, the proposed method reduced an average of more than 2.58% and 4% regression error on heterogeneous and homogeneous domains, respectively.

Multi-Network Feature Fusion Facial Emotion Recognition using Nonparametric Method with Augmentation

Facial expression emotion identification and prediction is one of the most difficult problems in computer science. Pre-processing and feature extraction are crucial components of the more conventional methods. For the purpose of emotion identification and prediction using 2D facial expressions, this study targets the Face Expression Recognition dataset and shows the real implementation or assessment of learning algorithms such as various CNNs. Due to its vast potential in areas like artificial intelligence, emotion detection from facial expressions has become an essential requirement. Many efforts have been done on the subject since it is both a challenging and fascinating challenge in computer vision. The focus of this study is on using a convolutional neural network supplemented with data to build a facial emotion recognition system. This method may use face images to identify seven fundamental emotions, including anger, contempt, fear, happiness, neutrality, sadness, and surprise. As well as improving upon the validation accuracy of current models, a convolutional neural network that takes use of data augmentation, feature fusion, and the NCA feature selection approach may assist solve some of their drawbacks. Researchers in this area are focused on improving computer predictions by creating methods to read and codify facial expressions. With deep learning's striking success, many architectures within the framework are being used to further the method's efficacy. We highlight the contributions dealt with, the architecture and databases used, and demonstrate the development by contrasting the offered approaches and the outcomes produced. The purpose of this study is to aid and direct future researchers in the subject by reviewing relevant recent studies and offering suggestions on how to further the field. An innovative feature-based transfer learning technique is created using the pre-trained networks MobileNetV2 and DenseNet-201. The suggested system's recognition rate is 75.31%, which is a significant improvement over the results of the prior feature fusion study.

Kernel Fisher Dictionary Transfer Learning

Dictionary learning is an efficient knowledge representation method that can learn the essential features of data. Traditional dictionary learning methods are difficult to obtain nonlinear information when processing large-scale and high-dimensional datasets. While most dictionary learning algorithms are based on the assumption that the training data and test data have the same feature distribution, which is not always true in practical applications. To address the above problems, we propose the Kernel Fisher Dictionary Transfer Learning (KFDTL) algorithm. First, we map each sample to high-dimensional space through kernel mapping and use any dictionary learning algorithm to learn the essential features. Then, the feature-based transfer learning method is performed to predict the labels of the target samples. This method includes three main contributions: (1) KFDTL constructs a discriminative Fisher embedding model to make the same class samples have similar coding coefficients; (2) Based on the relationship between profiles and atoms, KFDTL constructs an adaptive model that adapts source domain samples to target domain samples; (3) The kernel method is used to efficiently solve nonlinear problems. Experiments on a large number of public image datasets have proved the effectiveness of the proposed method. The source code of the proposed method is available at https://github.com/zzfan3/KFDTL .

The accuracy losing phenomenon in abrasive tool condition monitoring and a noval WMMC-JDA based data-driven method considered tool stochastic surface morphology

As a critically issue in the maintaining of desired part quality and manufacturing productivity, tool condition monitoring (TCM) problem is treated by multi data-driven machine learning methods with training and testing datasets generated by stochastic segmentation. For existing methods, it is assumed that samples collected from same type tools follow identical probability distribution. Different from regular tools, abrasive tools dissatisfy this assumption because of their stochastic surface morphologies. Therefore, traditional data-driven methods loss their accuracies in abrasive tool changing cases. In data driven TCM models, the macroscopic effect of abrasive tool micro-stochasticity is discovered. For the abrasive tool frequently changing cases, an abrasive tool condition monitoring (ATCM) method based on unsupervised domain adaptation is presented to deal with changes caused by stochastic surface morphology. In feature-based transfer learning, source and target domain datasets are projected into a transferred subspace to enhance the distribution alignment similarity of two domains. A weighted maximum margin criterion (WMMC) is adopted in the domain adaptation process to make the transformed samples in same class closing but segregated from those in different classes. With unsupervised domain adaptation mechanism and multilayer perceptron (MLP), the unsupervised domain adaptation is proposed to deal with traditional model invalidation after abrasive tool changes. The proposed method is verified by the robotic belt grinding experiment results. Compared with the existing data-driven method, the weighted maximum margin criterion joint distribution adaptation (WMMC-JDA) based ATCM method still maintains the predicting effectiveness after abrasive tool changes.

Data augmentation method for underwater acoustic target recognition based on underwater acoustic channel modeling and transfer learning

Data augmentation methods as a critical technique in deep learning have not been well studied in the underwater acoustic target recognition, which leads difficult for recognition models to cope with data scarcity and noise interference. This study proposes a data augmentation method based on underwater acoustic channel modeling and Transfer learning to address these challenges. A underwater acoustic channel modeling approach is proposed to generate the augmented signal. A feature-based transfer learning method is presented to narrow the distribution differences between augmented and observed data, and the noise is randomly added to enhance model robustness during training. Dataset acquired in a real-world scenario is used to verify the proposed methods. The proposed methods' effectiveness is proved by utilizing data augmentation in the model training process, which effectively improves the accuracy and noise robustness of the recognition model, especially when observed data is scarce.

Enhanced transfer learning method for rolling bearing fault diagnosis based on linear superposition network

Deep transfer learning is used to solve the problem of unsupervised intelligent fault diagnosis of rolling bearings. However, when the data distribution between two domains is different, the existing deep transfer learning models which only rely on the domain-invariant features are not enough to complete the target domain data learning. To solve this problem, an enhanced transfer learning method based on the linear superposition network is proposed for rolling bearing fault diagnosis. This method improves the structure of the one-dimensional convolutional neural network (1D-CNN) by constructing linear superposition convolution blocks. At the same time, the loss function of transfer learning is constructed by using the pseudo-label of the target domain from the network, which enhances the ability of rolling bearing fault feature extraction. Compared with the traditional feature-based transfer learning methods, the proposed enhanced transfer learning method based on the linear superposition network can make the network place more stress on the feature learning of the target domain. Experimental results on the Paderborn University (PU) dataset show that, compared with the improved deep adaptation network (DAN) model, the proposed method improves the average diagnosis accuracy by 21% on six transfer tasks, showing improved bearing fault diagnostic precision.

TERFDA: Tensor Embedding RF Domain Adaptation for varying noise interference

Radio frequency machine learning (RFML) can be loosely termed as a field that machine learning (ML) and deep learning (DL) techniques to applications related to wireless communications. However, traditional RFML basically assume that the data of training set and test set are independent and identically distributed and only a large number of labeled data can train a classification model which can effectively classify test set data. In other words, without enough training samples, it is impossible to learn an automatic modulation classifier that performs well in varying noise interference environment. Feature-based transfer learning minimizes the distribution difference between historical modulated signal data and new data by learning similarity-maximizing feature spaces. Therefore, in this paper, Dynamic Distribution Adaptation (DDA) is adopted to address the above challenges. We propose a Tensor Embedding RF Domain Adaptation (TERFDA) approach, which learns the latent subspace of the tensors formed by the time–frequency maps of the signals, so that use the multi-dimensional domain information of the signals to jointly learn the shared feature subspace of the source domain and the target domain, then perform DDA in the shared subspace. The experimental results show that under the modulated signal data, compared with the state-of-the-art DA algorithm, TERFDA has less requirements on the number of samples and categories, and has superior performance for confrontation the varying noise interference between source domain and target domain.

A Novel Tran_NAS Method for the Identification of Fe- and Mg-Deficient Pear Leaves from N- and P-Deficient Pear Leaf Data.

Trace element deficiency diagnosis plays a critical role in pear cultivation. However, high-quality diagnostic models are challenging to investigate, making it difficult to collect samples. Therefore, this manuscript developed a novel transfer learning method, named Tran_NAS, with a fine-tuning neural network that uses a neural architecture search (NAS) to transfer learning from nitrogen (N) and phosphorus (P) to iron (Fe) and magnesium (Mg) to diagnose pear leaf element deficiencies. The best accuracy of the transferred NAS model is 89.12%, which is 11% more than that of the model without the transfer of trace element-deficient samples. Meanwhile, Tran_NAS also has better performance on source datasets after comparing with different proportions of training sets. Finally, this manuscript summarizes the transfer model coincident characteristics, including the methods of batch normalization (BN) and dropout layers, which make the model more generalizable. This manuscript applies a symmetric homogeneous feature-based transfer learning method on NAS that is designed explicitly for near-infrared (NIR) data collected from nutrient-deficient pear leaves. The novel transfer learning method would be more effective for the micro-NIR spectrum of the nondestructive diagnosis.

Transfer fault diagnosis based on local maximum mean difference and K-means

Existing feature-based transfer learning methods have achieved great performance in the transfer fault diagnosis with unlabeled data. While most of them are global alignment methods based on maximum mean difference (MMD), which ignore the differences between different faults and pay little attention to the structural information in the unlabeled target samples. This paper proposes a transfer sparse auto-encoder (SAE) based on local maximum mean difference (LMMD) and K-means to solve the above problems. Firstly, we build a deep network based on SAE and LMMD for learning a common latent feature space where source and target subdomains are aligned. Subsequently, to fully explore the target domain information, we put forward the K-means-based method which can obtain final diagnosis results by synthesizing the source and target domain information in the latent feature space. Lastly, a case study is conducted to verify the robustness and effectiveness of the proposed methods. The experimental result demonstrates that the proposed methods outperform the MMD-based methods in the transfer fault diagnosis problem.

Creep rupture life prediction of nickel-based superalloys based on data fusion

Creep rupture life is a key performance parameter of nickel-based superalloys, which directly affects the engineering service behavior of components. In this study, a machine learning method based on data fusion was proposed to predict the properties of new alloys with the properties of existing alloys, or to predict other related properties of the same alloy, so as to solve the problem of accurate prediction of creep rupture life caused by the lack of accumulated data. Using the existing nickel-based superalloy creep rupture life data accumulated in the NIMS database, a creep rupture life prediction model was established using key alloy factor screening and feature-based transfer learning strategy (FS-SVR). The performance of GH4169D alloy was successfully predicted by the properties of GH4169 alloy, and the high temperature creep rupture life was predicted by the low temperature creep rupture life of GH4169 and GH4169D alloys, and the prediction accuracy reached more than 90%. The research results not only provide a fast method for predicting the creep properties of novel nickel-based superalloys, but also provide a reference case for data fusion to assist the research and development of new materials.

Novel fNIRS study on homogeneous symmetric feature-based transfer learning for brain\u2013computer interface

The brain–computer interface (BCI) provides an alternate means of communication between the brain and external devices by recognizing the brain activities and translating them into external commands. The functional Near-Infrared Spectroscopy (fNIRS) is becoming popular as a non-invasive modality for brain activity detection. The recent trends show that deep learning has significantly enhanced the performance of the BCI systems. But the inherent bottleneck for deep learning (in the domain of BCI) is the requirement of the vast amount of training data, lengthy recalibrating time, and expensive computational resources for training deep networks. Building a high-quality, large-scale annotated dataset for deep learning-based BCI systems is exceptionally tedious, complex, and expensive. This study investigates the novel application of transfer learning for fNIRS-based BCI to solve three objective functions (concerns), i.e., the problem of insufficient training data, reduced training time, and increased accuracy. We applied symmetric homogeneous feature-based transfer learning on convolutional neural network (CNN) designed explicitly for fNIRS data collected from twenty-six (26) participants performing the n-back task. The results suggested that the proposed method achieves the maximum saturated accuracy sooner and outperformed the traditional CNN model on averaged accuracy by 25.58% in the exact duration of training time, reducing the training time, recalibrating time, and computational resources.

Anomaly detection and condition monitoring of wind turbine gearbox based on LSTM-FS and transfer learning

To take full advantage of the limited monitoring data with fault information for operational state prediction in the case of the discrepancy in data distribution between the WTGs, a novel combined method is proposed based on the long short-term memory, fuzzy synthesis and feature-based transfer learning. After the statistical analysis and prediction of the monitoring indexes of two 2-MW WTGs with faulty information, an operational state calibration framework is proposed based on deep learning and fuzzy synthesis. Following this, three feature-based transfer learning methods are adopted to narrow the discrepancy among the data distribution of the WTGs. Correspondingly, feasibility verification of the proposed method is equally addressed. Case applications are performed using the actual monitoring data from No. 13 and 15 wind turbines of a wind farm in northern China. The results show that the operational state calibration framework can sensitively detect the potential fault information of the WTG in advance. Meanwhile, three transfer learning algorithms can effectively narrow the distance of the data distribution among the WTGs, and the classification accuracy can almost reach above 0.9. The proposed method can make full use of existing monitoring data with faulty information to predict the status of other WTGs.

AnomalyAdapters: Parameter-Efficient Multi-Anomaly Task Detection

The emergence of technological innovations brings sophisticated threats. Cyberattacks are increasing day by day aligned with these innovations and entails rapid solutions for defense mechanisms. These attacks may hinder enterprise operations or more importantly, interrupt critical infrastructure systems, that are essential to safety, security, and well-being of a society. Anomaly detection, as a protection step, is significant for ensuring a system security. Logs, which are accepted sources universally, are utilized in system health monitoring and intrusion detection systems. Recent developments in Natural Language Processing (NLP) studies show that contextual information decreases false-positives yield in detecting anomalous behaviors. Transformers and their adaptations to various language understanding tasks exemplify the enhanced ability to extract this information. Deep network based anomaly detection solutions use generally feature-based transfer learning methods. This type of learning presents a new set of weights for each log type. It is unfeasible and a redundant way considering various log sources. Also, a vague representation of model decisions prevents learning from threat data and improving model capability. In this paper, we propose AnomalyAdapters (AAs) which is an extensible multi-anomaly task detection model. It uses pretrained transformers’ variant to encode a log sequences and utilizes adapters to learn a log structure and anomaly types. Adapter-based approach collects contextual information, eliminates information loss in learning, and learns anomaly detection tasks from different log sources without overuse of parameters. Lastly, our work elucidates the decision making process of the proposed model on different log datasets to emphasize extraction of threat data via explainability experiments.

PF-TL: Payload Feature-Based Transfer Learning for Dealing with the Lack of Training Data

The number of studies on applying machine learning to cyber security has increased over the past few years. These studies, however, are facing difficulties with making themselves usable in the real world, mainly due to the lack of training data and reusability of a created model. While transfer learning seems like a solution to these problems, the number of studies in the field of intrusion detection is still insufficient. Therefore, this study proposes payload feature-based transfer learning as a solution to the lack of training data when applying machine learning to intrusion detection by using the knowledge from an already known domain. Firstly, it expands the extracting range of information from header to payload to accurately deliver the information by using an effective hybrid feature extraction method. Secondly, this study provides an improved optimization method for the extracted features to create a labeled dataset for a target domain. This proposal was validated on publicly available datasets, using three distinctive scenarios, and the results confirmed its usability in practice by increasing the accuracy of the training data created from the transfer learning by 30%, compared to that of the non-transfer learning method. In addition, we showed that this approach can help in identifying previously unknown attacks and reusing models from different domains.

Exploring diamondlike lattice thermal conductivity crystals via feature-based transfer learning

Ultrahigh lattice thermal conductivity materials hold great importance since they play a critical role in the thermal management of electronic and optical devices. Models using machine learning can search for materials with outstanding higher-order properties like thermal conductivity. However, the lack of sufficient data to train a model is a serious hurdle. Herein we show that big data can complement small data for accurate predictions when lower-order feature properties available in big data are selected properly and applied to transfer learning. The connection between the crystal information and thermal conductivity is directly built with a neural network by transferring descriptors acquired through a pretrained model for the feature property. Successful transfer learning shows the ability of extrapolative prediction and reveals descriptors for lattice anharmonicity. The resulting model is employed to screen over $60\phantom{\rule{0.16em}{0ex}}000$ compounds to identify novel crystals that can serve as alternatives to diamond. Even though most materials in the top list are superhard materials, we reveal that superhard property does not necessarily lead to high lattice thermal conductivity. Large hardness means high elastic constants and group velocity of phonons in the linear dispersion regime, but the lattice thermal conductivity is determined also by other important factors such as the phonon relaxation time. What is more, the average or maximum dipole polarizability and the van der Waals radius are revealed to be the leading descriptors among those that can also be qualitatively related to anharmonicity.

Transfer learning based robust automatic detection system for diabetic retinopathy grading

Diabetic retinopathy (DR) can be categorized on the basis of prolonged complication in the retinal blood vessels which may lead to severe blindness. Early stage prediction and diagnosis of DR requires regular eye examination to reduce the complications causing vision loss. Indicative significance of DR forecast and evaluation to help the ophthalmologists in standard screening has prompted the improvement of computerized DR recognition frameworks. This work focuses on automatic DR disease identification and its grading by the means of transfer learning approach using dynamic investigation. Our proposed approach utilizes deep neural network for feature extraction from fundus images and these features are further ensembled with supervised machine learning technique for DR grading. An optimized classification is achieved by applying an ensemble of convolution neural networks (CNNs) with statistical feature selection module and SVM classifier. The learning of classifier is achieved by the feature information transferred from CNN model to the SVM classifier, which results in remarkable performance of the learned models. Statistically optimized feature set utilized for transfer learning technique yields in the classification accuracy of 90.51% with proposed Prominent Feature-based Transfer Learning (PFTL) method employing Inception V3 model. The cost analysis of the proposed model provides a minimum cross-entropy loss of 0.295 consuming the time of 38 min 53 s, thus, maintaining a trade-off. The generalization ability of the proposed model is established by the performance assessment using latest IDRiD dataset that yields accuracy of 90.01% for Inception V3 network providing uniform outcomes for all the evaluation parameters. The diagnosis ability of the proposed transfer learning-based model is justified by comparing the proposed methods with the state-of-the-art methods. The optimized PFTL model (CNN + Statistical Analysis + SVM) outperforms other classification algorithms and provides the maximum accuracy improvement of 16.01% over the state-of-the-art techniques.

Transfer learning and its application research

Transfer learning relaxes the assumption’s limitation requirements of the independent and identical distribution of training data and test data in machine learning. It aims to help the target domain complete the learning task by learning one or more source domains similar to the target domain, solve the problem of scarcity of annotation data and enhance the model’s robustness and generalization performance. The article is a survey on the progress of transfer learning. According to “how to transfer”, transfer learning is divided into four categories: instance-based transfer learning, feature-based transfer learning, model-based transfer learning, and relation-based transfer learning. The paper introduces the basic assumptions, main research questions, common methods, related research of various transfer learning algorithms, and the application of the transfer learning. Finally, we try to point out future research trends.